Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fori loop simple case test test #7028

Support pin pr number in new .torch_pin (#6998)

cbbefa2
Select commit
Loading
Failed to load commit list.
Closed

Fori loop simple case test test #7028

Support pin pr number in new .torch_pin (#6998)
cbbefa2
Select commit
Loading
Failed to load commit list.
Google Cloud Build / dev-3-10-cuda-12-3 (tpu-pytorch-releases) succeeded May 1, 2024 in 17m 16s

Summary

Build Information

Trigger dev-3-10-cuda-12-3
Build 2827e3f6-1b6b-4b1a-ac95-31ef63be687e
Start 2024-05-01T12:42:37-07:00
Duration 16m30.369s
Status SUCCESS

Steps

Step Status Duration
git_fetch SUCCESS 9.128s
git_checkout SUCCESS 599ms
build_development_docker_image SUCCESS 8m21.011s
push_development_docker_image SUCCESS 7m56.424s

Details

starting build "2827e3f6-1b6b-4b1a-ac95-31ef63be687e"

FETCHSOURCE
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint: 
hint: 	git config --global init.defaultBranch <name>
hint: 
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint: 
hint: 	git branch -m <name>
Initialized empty Git repository in /workspace/.git/
From https://github.com/pytorch/xla
 * branch            cbbefa2c129ae48c6e868b9eeb4985b6e562dc1c -> FETCH_HEAD
HEAD is now at cbbefa2 Support pin pr number in new .torch_pin (#6998)
BUILD
Starting Step #0 - "git_fetch"
Step #0 - "git_fetch": Already have image (with digest): gcr.io/cloud-builders/git
Step #0 - "git_fetch": From https://github.com/pytorch/xla
Step #0 - "git_fetch":  * [new branch]          01172023_pin_update     -> origin/01172023_pin_update
Step #0 - "git_fetch":  * [new branch]          2-0-python-3-10         -> origin/2-0-python-3-10
Step #0 - "git_fetch":  * [new branch]          4483-improve-resnet50-data-loading-performance -> origin/4483-improve-resnet50-data-loading-performance
Step #0 - "git_fetch":  * [new branch]          5d-unit-test            -> origin/5d-unit-test
Step #0 - "git_fetch":  * [new branch]          AddVimToReleasePackage  -> origin/AddVimToReleasePackage
Step #0 - "git_fetch":  * [new branch]          FixTransferToServer     -> origin/FixTransferToServer
Step #0 - "git_fetch":  * [new branch]          JackCaaoG/spmd_device_python_test -> origin/JackCaaoG/spmd_device_python_test
Step #0 - "git_fetch":  * [new branch]          JackCao/Disable_hlo_dump_test_on_tpu -> origin/JackCao/Disable_hlo_dump_test_on_tpu
Step #0 - "git_fetch":  * [new branch]          JackCao/Dynamo_sync_input_counter -> origin/JackCao/Dynamo_sync_input_counter
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_1 -> origin/JackCao/cherry_pick_dynamo_1
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_2 -> origin/JackCao/cherry_pick_dynamo_2
Step #0 - "git_fetch":  * [new branch]          JackCao/cherry_pick_dynamo_3 -> origin/JackCao/cherry_pick_dynamo_3
Step #0 - "git_fetch":  * [new branch]          JackCao/spmd_python_device -> origin/JackCao/spmd_python_device
Step #0 - "git_fetch":  * [new branch]          JackCao/unordered_sharding_spec -> origin/JackCao/unordered_sharding_spec
Step #0 - "git_fetch":  * [new branch]          JackCao/update_to_2.0   -> origin/JackCao/update_to_2.0
Step #0 - "git_fetch":  * [new branch]          JackCao/update_to_223   -> origin/JackCao/update_to_223
Step #0 - "git_fetch":  * [new branch]          JackCao/workaround_einsum_faketensor -> origin/JackCao/workaround_einsum_faketensor
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-1        -> origin/JackCaoG-patch-1
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-2        -> origin/JackCaoG-patch-2
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-3        -> origin/JackCaoG-patch-3
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-4        -> origin/JackCaoG-patch-4
Step #0 - "git_fetch":  * [new branch]          JackCaoG-patch-5        -> origin/JackCaoG-patch-5
Step #0 - "git_fetch":  * [new branch]          JackCaoG-remove-pin     -> origin/JackCaoG-remove-pin
Step #0 - "git_fetch":  * [new branch]          JackCaoG/1_13_revert_addcmul -> origin/JackCaoG/1_13_revert_addcmul
Step #0 - "git_fetch":  * [new branch]          JackCaoG/2.0_patch      -> origin/JackCaoG/2.0_patch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/AddDynamoSpmdMetrics -> origin/JackCaoG/AddDynamoSpmdMetrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/CI_build_cpp_test -> origin/JackCaoG/CI_build_cpp_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Dyanmo_execute_async -> origin/JackCaoG/Dyanmo_execute_async
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Dynamo_training_test -> origin/JackCaoG/Dynamo_training_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/PJRT_wait_buffer_sync -> origin/JackCaoG/PJRT_wait_buffer_sync
Step #0 - "git_fetch":  * [new branch]          JackCaoG/SPMD_execute_metrics -> origin/JackCaoG/SPMD_execute_metrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/SPMD_int_to_cpu -> origin/JackCaoG/SPMD_int_to_cpu
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Update_short_metrics -> origin/JackCaoG/Update_short_metrics
Step #0 - "git_fetch":  * [new branch]          JackCaoG/Update_wheel_docker_to_table -> origin/JackCaoG/Update_wheel_docker_to_table
Step #0 - "git_fetch":  * [new branch]          JackCaoG/XLAShardedTensor_torch_dispatch -> origin/JackCaoG/XLAShardedTensor_torch_dispatch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_2_1_py311_build -> origin/JackCaoG/add_2_1_py311_build
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_dynamo_test_counter_check -> origin/JackCaoG/add_dynamo_test_counter_check
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_dynamo_test_to_tpu_ci -> origin/JackCaoG/add_dynamo_test_to_tpu_ci
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_gpu_doc    -> origin/JackCaoG/add_gpu_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_gpu_doc_readme -> origin/JackCaoG/add_gpu_doc_readme
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_test_dynamo_graph_dump -> origin/JackCaoG/add_test_dynamo_graph_dump
Step #0 - "git_fetch":  * [new branch]          JackCaoG/add_test_print -> origin/JackCaoG/add_test_print
Step #0 - "git_fetch":  * [new branch]          JackCaoG/api_init_runtime -> origin/JackCaoG/api_init_runtime
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backout_22_mesh_spmd -> origin/JackCaoG/backout_22_mesh_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_allgather_out -> origin/JackCaoG/backport_22_allgather_out
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_diagonal_scatter -> origin/JackCaoG/backport_22_diagonal_scatter
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_doc -> origin/JackCaoG/backport_22_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_reduce_scatter -> origin/JackCaoG/backport_22_reduce_scatter
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_reducescatter_out -> origin/JackCaoG/backport_22_reducescatter_out
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_scalar -> origin/JackCaoG/backport_22_scalar
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_22_transfer_gpu_scalar -> origin/JackCaoG/backport_22_transfer_gpu_scalar
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_aws_zero1 -> origin/JackCaoG/backport_aws_zero1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_dynamo_cpu_move -> origin/JackCaoG/backport_dynamo_cpu_move
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_dynamo_doc_eval -> origin/JackCaoG/backport_dynamo_doc_eval
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_flash_attention_dynamo -> origin/JackCaoG/backport_flash_attention_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_frame_count -> origin/JackCaoG/backport_frame_count
Step #0 - "git_fetch":  * [new branch]          JackCaoG/backport_oerserve_rng -> origin/JackCaoG/backport_oerserve_rng
Step #0 - "git_fetch":  * [new branch]          JackCaoG/barrier_comment -> origin/JackCaoG/barrier_comment
Step #0 - "git_fetch":  * [new branch]          JackCaoG/better_dynamo_error_message -> origin/JackCaoG/better_dynamo_error_message
Step #0 - "git_fetch":  * [new branch]          JackCaoG/buffer_donor_custom_op -> origin/JackCaoG/buffer_donor_custom_op
Step #0 - "git_fetch":  * [new branch]          JackCaoG/build_audio_whl -> origin/JackCaoG/build_audio_whl
Step #0 - "git_fetch":  * [new branch]          JackCaoG/bump+gpu_coverage_timeout -> origin/JackCaoG/bump+gpu_coverage_timeout
Step #0 - "git_fetch":  * [new branch]          JackCaoG/bump_build_timeout -> origin/JackCaoG/bump_build_timeout
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cache_sharded_placeholder_dynamo -> origin/JackCaoG/cache_sharded_placeholder_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_pick_dynamo_test -> origin/JackCaoG/cherry_pick_dynamo_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_pick_frame_count -> origin/JackCaoG/cherry_pick_frame_count
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cherry_picl_xrt_crash_2.1 -> origin/JackCaoG/cherry_picl_xrt_crash_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clang-format-11 -> origin/JackCaoG/clang-format-11
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clang-format-11-2 -> origin/JackCaoG/clang-format-11-2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clean_up_gettensor -> origin/JackCaoG/clean_up_gettensor
Step #0 - "git_fetch":  * [new branch]          JackCaoG/clean_up_use_virtual_device -> origin/JackCaoG/clean_up_use_virtual_device
Step #0 - "git_fetch":  * [new branch]          JackCaoG/cleanup_docs   -> origin/JackCaoG/cleanup_docs
Step #0 - "git_fetch":  * [new branch]          JackCaoG/colab_whl_abi  -> origin/JackCaoG/colab_whl_abi
Step #0 - "git_fetch":  * [new branch]          JackCaoG/colab_whl_abi_2 -> origin/JackCaoG/colab_whl_abi_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/data_to_string -> origin/JackCaoG/data_to_string
Step #0 - "git_fetch":  * [new branch]          JackCaoG/debug_fix      -> origin/JackCaoG/debug_fix
Step #0 - "git_fetch":  * [new branch]          JackCaoG/debug_tool     -> origin/JackCaoG/debug_tool
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_coverage -> origin/JackCaoG/disable_coverage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_dynamo_different_shape_test -> origin/JackCaoG/disable_dynamo_different_shape_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test -> origin/JackCaoG/disable_flaky_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test_2 -> origin/JackCaoG/disable_flaky_test_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_flaky_test_3 -> origin/JackCaoG/disable_flaky_test_3
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_more_dynamic -> origin/JackCaoG/disable_more_dynamic
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_more_dynamic_test -> origin/JackCaoG/disable_more_dynamic_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_pjrt_computation_client_test -> origin/JackCaoG/disable_pjrt_computation_client_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disable_test_xla_sharding -> origin/JackCaoG/disable_test_xla_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/disbale_docker_sandbox -> origin/JackCaoG/disbale_docker_sandbox
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dump_output_sharding -> origin/JackCaoG/dump_output_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_2.0_cherry_pick -> origin/JackCaoG/dynamo_2.0_cherry_pick
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_aliasing_2 -> origin/JackCaoG/dynamo_aliasing_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_debug_message -> origin/JackCaoG/dynamo_debug_message
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_different_graph -> origin/JackCaoG/dynamo_different_graph
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_do_not_markstep_to_sync_input -> origin/JackCaoG/dynamo_do_not_markstep_to_sync_input
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_doc     -> origin/JackCaoG/dynamo_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_dump_tensor_file -> origin/JackCaoG/dynamo_dump_tensor_file
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_note_backport_2.1 -> origin/JackCaoG/dynamo_note_backport_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_only_warm_up_cache -> origin/JackCaoG/dynamo_only_warm_up_cache
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_optimizer -> origin/JackCaoG/dynamo_optimizer
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_reduce_execute -> origin/JackCaoG/dynamo_reduce_execute
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_rng_seed_without_mark_step -> origin/JackCaoG/dynamo_rng_seed_without_mark_step
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd    -> origin/JackCaoG/dynamo_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd_input_sharding_change -> origin/JackCaoG/dynamo_spmd_input_sharding_change
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_spmd_optimization -> origin/JackCaoG/dynamo_spmd_optimization
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_sync_input -> origin/JackCaoG/dynamo_sync_input
Step #0 - "git_fetch":  * [new branch]          JackCaoG/dynamo_warning -> origin/JackCaoG/dynamo_warning
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_cumsum_long -> origin/JackCaoG/enable_cumsum_long
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_dynamo_upstream_test -> origin/JackCaoG/enable_dynamo_upstream_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_fsdp_gpu_test -> origin/JackCaoG/enable_fsdp_gpu_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_mp_test_on_tpuci -> origin/JackCaoG/enable_mp_test_on_tpuci
Step #0 - "git_fetch":  * [new branch]          JackCaoG/enable_reduce_scatter_test_gpu -> origin/JackCaoG/enable_reduce_scatter_test_gpu
Step #0 - "git_fetch":  * [new branch]          JackCaoG/example_torch_pin_pr -> origin/JackCaoG/example_torch_pin_pr
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_cc_linter_once -> origin/JackCaoG/fix_cc_linter_once
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_deprecated_test_oepration_api -> origin/JackCaoG/fix_deprecated_test_oepration_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_diagonal_scatter_dim -> origin/JackCaoG/fix_diagonal_scatter_dim
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dyanmo_additional_tracing -> origin/JackCaoG/fix_dyanmo_additional_tracing
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo     -> origin/JackCaoG/fix_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_crash -> origin/JackCaoG/fix_dynamo_crash
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_crash_profiler -> origin/JackCaoG/fix_dynamo_crash_profiler
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_inplace -> origin/JackCaoG/fix_dynamo_inplace
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_input_sharding_test -> origin/JackCaoG/fix_dynamo_input_sharding_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_dynamo_text_allclose -> origin/JackCaoG/fix_dynamo_text_allclose
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_full_dtype -> origin/JackCaoG/fix_full_dtype
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_hlo_dump   -> origin/JackCaoG/fix_hlo_dump
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_hlo_dump_spmd -> origin/JackCaoG/fix_hlo_dump_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_log_spam   -> origin/JackCaoG/fix_log_spam
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_mark_step_on_spmd_device -> origin/JackCaoG/fix_mark_step_on_spmd_device
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_mp_save_test -> origin/JackCaoG/fix_mp_save_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_send_cpu_data -> origin/JackCaoG/fix_send_cpu_data
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_spmd_buff_is_null -> origin/JackCaoG/fix_spmd_buff_is_null
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_tpu_ci_accuracy -> origin/JackCaoG/fix_tpu_ci_accuracy
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_typo       -> origin/JackCaoG/fix_typo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_vision_build -> origin/JackCaoG/fix_vision_build
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fix_warning_message -> origin/JackCaoG/fix_warning_message
Step #0 - "git_fetch":  * [new branch]          JackCaoG/flash_attention_custom_op -> origin/JackCaoG/flash_attention_custom_op
Step #0 - "git_fetch":  * [new branch]          JackCaoG/flash_attention_test_precision -> origin/JackCaoG/flash_attention_test_precision
Step #0 - "git_fetch":  * [new branch]          JackCaoG/fsdp_autowrap_test -> origin/JackCaoG/fsdp_autowrap_test
Step #0 - "git_fetch":  * [new branch]          JackCaoG/gpu_build_pytorch_cuda -> origin/JackCaoG/gpu_build_pytorch_cuda
Step #0 - "git_fetch":  * [new branch]          JackCaoG/graph_hash_debug -> origin/JackCaoG/graph_hash_debug
Step #0 - "git_fetch":  * [new branch]          JackCaoG/handle_dynamo_fn_without_arg -> origin/JackCaoG/handle_dynamo_fn_without_arg
Step #0 - "git_fetch":  * [new branch]          JackCaoG/improve_data_tostring -> origin/JackCaoG/improve_data_tostring
Step #0 - "git_fetch":  * [new branch]          JackCaoG/increase_timeout_docker -> origin/JackCaoG/increase_timeout_docker
Step #0 - "git_fetch":  * [new branch]          JackCaoG/ir_value_sharding -> origin/JackCaoG/ir_value_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/lower_dropout_backward -> origin/JackCaoG/lower_dropout_backward
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_op_lowering_guide -> origin/JackCaoG/move_op_lowering_guide
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_pin_2_0   -> origin/JackCaoG/move_pin_2_0
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_pjrt_lock_location -> origin/JackCaoG/move_pjrt_lock_location
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_return_future -> origin/JackCaoG/move_return_future
Step #0 - "git_fetch":  * [new branch]          JackCaoG/move_test_grad_checkpoint_to_tpu -> origin/JackCaoG/move_test_grad_checkpoint_to_tpu
Step #0 - "git_fetch":  * [new branch]          JackCaoG/new_alias_api  -> origin/JackCaoG/new_alias_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/nightly_build_python -> origin/JackCaoG/nightly_build_python
Step #0 - "git_fetch":  * [new branch]          JackCaoG/parallel_spmd  -> origin/JackCaoG/parallel_spmd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pin_expecttest -> origin/JackCaoG/pin_expecttest
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pin_update_20230808 -> origin/JackCaoG/pin_update_20230808
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_5 -> origin/JackCaoG/pt_xla_debug_5
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_dynamo_execute -> origin/JackCaoG/pt_xla_debug_dynamo_execute
Step #0 - "git_fetch":  * [new branch]          JackCaoG/pt_xla_debug_max_frame -> origin/JackCaoG/pt_xla_debug_max_frame
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r20_backportdoc -> origin/JackCaoG/r20_backportdoc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_ABI_0     -> origin/JackCaoG/r2_0_ABI_0
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_back_port_profile_flag -> origin/JackCaoG/r2_0_back_port_profile_flag
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_backport_loader_thread -> origin/JackCaoG/r2_0_backport_loader_thread
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_backport_loader_thread_2 -> origin/JackCaoG/r2_0_backport_loader_thread_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/r2_0_cherry_pick_defer_init -> origin/JackCaoG/r2_0_cherry_pick_defer_init
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_pin     -> origin/JackCaoG/remove_pin
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_reshape_again -> origin/JackCaoG/remove_reshape_again
Step #0 - "git_fetch":  * [new branch]          JackCaoG/remove_reshape_lowering -> origin/JackCaoG/remove_reshape_lowering
Step #0 - "git_fetch":  * [new branch]          JackCaoG/restrict_mark_sharding -> origin/JackCaoG/restrict_mark_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/revert_addcmul -> origin/JackCaoG/revert_addcmul
Step #0 - "git_fetch":  * [new branch]          JackCaoG/revert_protobuf_tf_patch -> origin/JackCaoG/revert_protobuf_tf_patch
Step #0 - "git_fetch":  * [new branch]          JackCaoG/shard_device_data_ir -> origin/JackCaoG/shard_device_data_ir
Step #0 - "git_fetch":  * [new branch]          JackCaoG/skip_partitioner -> origin/JackCaoG/skip_partitioner
Step #0 - "git_fetch":  * [new branch]          JackCaoG/skip_tpu_test_precision -> origin/JackCaoG/skip_tpu_test_precision
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_aliasing  -> origin/JackCaoG/spmd_aliasing
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_dynamo_sharding_changed -> origin/JackCaoG/spmd_dynamo_sharding_changed
Step #0 - "git_fetch":  * [new branch]          JackCaoG/spmd_incorrect_result -> origin/JackCaoG/spmd_incorrect_result
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tensor_debugging_api -> origin/JackCaoG/tensor_debugging_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tensor_with_xla_data_sharding -> origin/JackCaoG/tensor_with_xla_data_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaoG/test_torch_compile_usage -> origin/JackCaoG/test_torch_compile_usage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tf_update_20230201 -> origin/JackCaoG/tf_update_20230201
Step #0 - "git_fetch":  * [new branch]          JackCaoG/tracing_timer  -> origin/JackCaoG/tracing_timer
Step #0 - "git_fetch":  * [new branch]          JackCaoG/trouble_shooting_update -> origin/JackCaoG/trouble_shooting_update
Step #0 - "git_fetch":  * [new branch]          JackCaoG/truncate_python_stack -> origin/JackCaoG/truncate_python_stack
Step #0 - "git_fetch":  * [new branch]          JackCaoG/unordered_partial_sharding_spec -> origin/JackCaoG/unordered_partial_sharding_spec
Step #0 - "git_fetch":  * [new branch]          JackCaoG/updateDocker   -> origin/JackCaoG/updateDocker
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_2_0_ABI -> origin/JackCaoG/update_2_0_ABI
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_API_GUIDE -> origin/JackCaoG/update_API_GUIDE
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE -> origin/JackCaoG/update_XLA_IR_SHAPE_CACHE_SIZE
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_build_guide -> origin/JackCaoG/update_build_guide
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_colab_2_10_whl -> origin/JackCaoG/update_colab_2_10_whl
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_doc1    -> origin/JackCaoG/update_doc1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_doc_remove_xrt -> origin/JackCaoG/update_doc_remove_xrt
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_docker_instruction -> origin/JackCaoG/update_docker_instruction
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_backend_name -> origin/JackCaoG/update_dynamo_backend_name
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_backend_nograd -> origin/JackCaoG/update_dynamo_backend_nograd
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_counter_check -> origin/JackCaoG/update_dynamo_counter_check
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_doc -> origin/JackCaoG/update_dynamo_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_doc_to_remove_eval -> origin/JackCaoG/update_dynamo_doc_to_remove_eval
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_number_2.1 -> origin/JackCaoG/update_dynamo_number_2.1
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_readme -> origin/JackCaoG/update_dynamo_readme
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_save_graph_path -> origin/JackCaoG/update_dynamo_save_graph_path
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_torch_equal -> origin/JackCaoG/update_dynamo_torch_equal
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_dynamo_training_compile_count -> origin/JackCaoG/update_dynamo_training_compile_count
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_dev -> origin/JackCaoG/update_gpu_dev
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_doc -> origin/JackCaoG/update_gpu_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_wheel_link -> origin/JackCaoG/update_gpu_wheel_link
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_gpu_wheellink -> origin/JackCaoG/update_gpu_wheellink
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_pjrt_future -> origin/JackCaoG/update_pjrt_future
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_readme_2 -> origin/JackCaoG/update_readme_2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_set_default_tensor_type -> origin/JackCaoG/update_set_default_tensor_type
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_tf_2023018 -> origin/JackCaoG/update_tf_2023018
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_to_2.2  -> origin/JackCaoG/update_to_2.2
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_to_223  -> origin/JackCaoG/update_to_223
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_torch_compile_usage -> origin/JackCaoG/update_torch_compile_usage
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_trouble_shotting_doc -> origin/JackCaoG/update_trouble_shotting_doc
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_troubleshotting_with_sanitycheck -> origin/JackCaoG/update_troubleshotting_with_sanitycheck
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_wheel_docker_format -> origin/JackCaoG/update_wheel_docker_format
Step #0 - "git_fetch":  * [new branch]          JackCaoG/update_xla_pin_040922024 -> origin/JackCaoG/update_xla_pin_040922024
Step #0 - "git_fetch":  * [new branch]          JackCaoG/use_new_alias_api -> origin/JackCaoG/use_new_alias_api
Step #0 - "git_fetch":  * [new branch]          JackCaoG/weak_ref_dynamo -> origin/JackCaoG/weak_ref_dynamo
Step #0 - "git_fetch":  * [new branch]          JackCaoG/xrt_cherry_pick_07_10 -> origin/JackCaoG/xrt_cherry_pick_07_10
Step #0 - "git_fetch":  * [new branch]          JackCaoG/xrt_cherrypick_0721 -> origin/JackCaoG/xrt_cherrypick_0721
Step #0 - "git_fetch":  * [new branch]          JackCaog/dynamo_input_sharding -> origin/JackCaog/dynamo_input_sharding
Step #0 - "git_fetch":  * [new branch]          JackCaog/quick_spmd_fix -> origin/JackCaog/quick_spmd_fix
Step #0 - "git_fetch":  * [new branch]          JackCaog/spmd_device_default -> origin/JackCaog/spmd_device_default
Step #0 - "git_fetch":  * [new branch]          Liyang90-patch-1        -> origin/Liyang90-patch-1
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-1       -> origin/ManfeiBai-patch-1
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-10      -> origin/ManfeiBai-patch-10
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-11      -> origin/ManfeiBai-patch-11
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-12      -> origin/ManfeiBai-patch-12
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-13      -> origin/ManfeiBai-patch-13
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-14      -> origin/ManfeiBai-patch-14
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-15      -> origin/ManfeiBai-patch-15
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-16      -> origin/ManfeiBai-patch-16
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-17      -> origin/ManfeiBai-patch-17
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-18      -> origin/ManfeiBai-patch-18
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-19      -> origin/ManfeiBai-patch-19
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-2       -> origin/ManfeiBai-patch-2
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-20      -> origin/ManfeiBai-patch-20
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-21      -> origin/ManfeiBai-patch-21
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-22      -> origin/ManfeiBai-patch-22
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-23      -> origin/ManfeiBai-patch-23
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-24      -> origin/ManfeiBai-patch-24
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-25      -> origin/ManfeiBai-patch-25
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-26      -> origin/ManfeiBai-patch-26
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-27      -> origin/ManfeiBai-patch-27
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-28      -> origin/ManfeiBai-patch-28
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-29      -> origin/ManfeiBai-patch-29
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-3       -> origin/ManfeiBai-patch-3
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-30      -> origin/ManfeiBai-patch-30
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-31      -> origin/ManfeiBai-patch-31
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-32      -> origin/ManfeiBai-patch-32
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-33      -> origin/ManfeiBai-patch-33
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-34      -> origin/ManfeiBai-patch-34
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-35      -> origin/ManfeiBai-patch-35
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-36      -> origin/ManfeiBai-patch-36
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-37      -> origin/ManfeiBai-patch-37
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-38      -> origin/ManfeiBai-patch-38
Step #0 - "git_fetch":  * [new branch]          ManfeiBai-patch-39      -> origin/ManfeiBai-patch-39
Step #0 - "gi
...
[Logs truncated due to log size limitations. For full logs, see https://console.cloud.google.com/cloud-build/builds;region=us-central1/2827e3f6-1b6b-4b1a-ac95-31ef63be687e?project=1001674285173.]
...
bi3-manylinux_2_28_x86_64.whl (4.6 MB)
Step #2 - "build_development_docker_image":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.6/4.6 MB 74.9 MB/s eta 0:00:00
Step #2 - "build_development_docker_image": Collecting resolvelib<1.1.0,>=0.5.3
Step #2 - "build_development_docker_image":   Downloading resolvelib-1.0.1-py2.py3-none-any.whl (17 kB)
Step #2 - "build_development_docker_image": Collecting packaging
Step #2 - "build_development_docker_image":   Downloading packaging-24.0-py3-none-any.whl (53 kB)
Step #2 - "build_development_docker_image":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.5/53.5 kB 10.0 MB/s eta 0:00:00
Step #2 - "build_development_docker_image": Collecting PyYAML>=5.1
Step #2 - "build_development_docker_image":   Downloading PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
Step #2 - "build_development_docker_image":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 705.5/705.5 kB 60.9 MB/s eta 0:00:00
Step #2 - "build_development_docker_image": Collecting MarkupSafe>=2.0
Step #2 - "build_development_docker_image":   Downloading MarkupSafe-2.1.5-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (25 kB)
Step #2 - "build_development_docker_image": Collecting cffi>=1.12
Step #2 - "build_development_docker_image":   Downloading cffi-1.16.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (443 kB)
Step #2 - "build_development_docker_image":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 443.9/443.9 kB 50.3 MB/s eta 0:00:00
Step #2 - "build_development_docker_image": Collecting pycparser
Step #2 - "build_development_docker_image":   Downloading pycparser-2.22-py3-none-any.whl (117 kB)
Step #2 - "build_development_docker_image":      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 117.6/117.6 kB 21.7 MB/s eta 0:00:00
Step #2 - "build_development_docker_image": Installing collected packages: resolvelib, PyYAML, pycparser, packaging, MarkupSafe, jinja2, cffi, cryptography, ansible-core, ansible
Step #2 - "build_development_docker_image": Successfully installed MarkupSafe-2.1.5 PyYAML-6.0.1 ansible-9.5.1 ansible-core-2.16.6 cffi-1.16.0 cryptography-42.0.5 jinja2-3.1.3 packaging-24.0 pycparser-2.22 resolvelib-1.0.1
Step #2 - "build_development_docker_image": �[91mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
Step #2 - "build_development_docker_image": �[0m�[91m
Step #2 - "build_development_docker_image": [notice] A new release of pip is available: 23.0.1 -> 24.0
Step #2 - "build_development_docker_image": [notice] To update, run: pip install --upgrade pip
Step #2 - "build_development_docker_image": �[0mRemoving intermediate container ae378b76f877
Step #2 - "build_development_docker_image":  ---> 84a10cb21346
Step #2 - "build_development_docker_image": Step 5/10 : COPY . /ansible
Step #2 - "build_development_docker_image":  ---> 193c7290ae08
Step #2 - "build_development_docker_image": Step 6/10 : WORKDIR /ansible
Step #2 - "build_development_docker_image":  ---> Running in cbeb774e3e27
Step #2 - "build_development_docker_image": Removing intermediate container cbeb774e3e27
Step #2 - "build_development_docker_image":  ---> 5fcfc99d541b
Step #2 - "build_development_docker_image": Step 7/10 : ENV TAGS="bazel,configure_env,install_deps"
Step #2 - "build_development_docker_image":  ---> Running in 1c8bc05a3e2a
Step #2 - "build_development_docker_image": Removing intermediate container 1c8bc05a3e2a
Step #2 - "build_development_docker_image":  ---> 175e28ade48b
Step #2 - "build_development_docker_image": Step 8/10 : ARG ansible_vars
Step #2 - "build_development_docker_image":  ---> Running in c60b7e291925
Step #2 - "build_development_docker_image": Removing intermediate container c60b7e291925
Step #2 - "build_development_docker_image":  ---> e766f5d76d88
Step #2 - "build_development_docker_image": Step 9/10 : RUN ansible-playbook playbook.yaml -e "stage=build" -e "${ansible_vars}" --tags "${TAGS}"
Step #2 - "build_development_docker_image":  ---> Running in d5ce950de2c8
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": PLAY [Install build dependencies] **********************************************
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [Gathering Facts] *********************************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:45 +0000 (0:00:00.027)       0:00:00.027 ********* 
Step #2 - "build_development_docker_image": ok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [Include vars from config files] ******************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:46 +0000 (0:00:01.027)       0:00:01.055 ********* 
Step #2 - "build_development_docker_image": ok: [localhost] => (item=vars.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=cuda_deps.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=apt.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=pip.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=env.yaml)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Download bazelisk v1.15.0] ***************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:46 +0000 (0:00:00.072)       0:00:01.127 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Tests] ***********************************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:47 +0000 (0:00:00.938)       0:00:02.065 ********* 
Step #2 - "build_development_docker_image": included: /ansible/roles/bazel/tasks/tests.yaml for localhost
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Bazel --version runs succesfully] ********************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:47 +0000 (0:00:00.029)       0:00:02.094 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Add apt keys] *********************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:48 +0000 (0:00:00.692)       0:00:02.787 ********* 
Step #2 - "build_development_docker_image": changed: [localhost] => (item={'url': 'https://apt.llvm.org/llvm-snapshot.gpg.key', 'keyring': '/usr/share/keyrings/llvm.pgp'})
Step #2 - "build_development_docker_image": changed: [localhost] => (item={'url': 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub', 'keyring': '/usr/share/keyrings/cuda.pgp'})
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Add apt repositories into sources list] *******************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:43:49 +0000 (0:00:00.920)       0:00:03.708 ********* 
Step #2 - "build_development_docker_image": changed: [localhost] => (item=deb [signed-by=/usr/share/keyrings/llvm.pgp] http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-17 main)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=deb-src [signed-by=/usr/share/keyrings/llvm.pgp] http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-17 main)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=deb [signed-by=/usr/share/keyrings/cuda.pgp] https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/ /)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Update apt cache] *****************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:44:00 +0000 (0:00:11.783)       0:00:15.491 ********* 
Step #2 - "build_development_docker_image": ok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install apt packages] *************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:44:03 +0000 (0:00:03.106)       0:00:18.598 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install pip packages] *************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:48:59 +0000 (0:04:55.635)       0:05:14.233 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install pip packages without deps (--no-deps)] ************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:49:39 +0000 (0:00:40.166)       0:05:54.400 ********* 
Step #2 - "build_development_docker_image": �[91m[WARNING]: No valid name or requirements file found.
Step #2 - "build_development_docker_image": �[0mok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [configure_env : Append environment variables required during runtime to ~/.bashrc] ***
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:49:40 +0000 (0:00:00.319)       0:05:54.720 ********* 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=CC) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=CXX) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=LD_LIBRARY_PATH) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=TF_CUDA_COMPUTE_CAPABILITIES) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=XLA_CUDA) 
Step #2 - "build_development_docker_image": skipping: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [configure_env : Append environment variables required during runtime to ~/.zshrc] ***
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:49:40 +0000 (0:00:00.064)       0:05:54.784 ********* 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=CC) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=CXX) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=LD_LIBRARY_PATH) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=TF_CUDA_COMPUTE_CAPABILITIES) 
Step #2 - "build_development_docker_image": skipping: [localhost] => (item=XLA_CUDA) 
Step #2 - "build_development_docker_image": skipping: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": PLAY RECAP *********************************************************************
Step #2 - "build_development_docker_image": localhost                  : ok=11   changed=6    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:49:40 +0000 (0:00:00.085)       0:05:54.870 ********* 
Step #2 - "build_development_docker_image": =============================================================================== 
Step #2 - "build_development_docker_image": install_deps : Install apt packages ----------------------------------- 295.64s
Step #2 - "build_development_docker_image": install_deps : Install pip packages ------------------------------------ 40.17s
Step #2 - "build_development_docker_image": install_deps : Add apt repositories into sources list ------------------ 11.78s
Step #2 - "build_development_docker_image": install_deps : Update apt cache ----------------------------------------- 3.11s
Step #2 - "build_development_docker_image": Gathering Facts --------------------------------------------------------- 1.03s
Step #2 - "build_development_docker_image": bazel : Download bazelisk v1.15.0 --------------------------------------- 0.94s
Step #2 - "build_development_docker_image": install_deps : Add apt keys --------------------------------------------- 0.92s
Step #2 - "build_development_docker_image": bazel : Bazel --version runs succesfully -------------------------------- 0.69s
Step #2 - "build_development_docker_image": install_deps : Install pip packages without deps (--no-deps) ------------ 0.32s
Step #2 - "build_development_docker_image": configure_env : Append environment variables required during runtime to ~/.zshrc --- 0.09s
Step #2 - "build_development_docker_image": Include vars from config files ------------------------------------------ 0.07s
Step #2 - "build_development_docker_image": configure_env : Append environment variables required during runtime to ~/.bashrc --- 0.06s
Step #2 - "build_development_docker_image": bazel : Tests ----------------------------------------------------------- 0.03s
Step #2 - "build_development_docker_image": Removing intermediate container d5ce950de2c8
Step #2 - "build_development_docker_image":  ---> 75d8733f33ab
Step #2 - "build_development_docker_image": Step 10/10 : RUN ansible-playbook playbook.yaml -e "stage=release" -e "${ansible_vars}" --tags "${TAGS}"
Step #2 - "build_development_docker_image":  ---> Running in 8666ef6ebe69
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": PLAY [Install build dependencies] **********************************************
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [Gathering Facts] *********************************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:54 +0000 (0:00:00.024)       0:00:00.024 ********* 
Step #2 - "build_development_docker_image": ok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [Include vars from config files] ******************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:55 +0000 (0:00:01.121)       0:00:01.146 ********* 
Step #2 - "build_development_docker_image": ok: [localhost] => (item=vars.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=cuda_deps.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=apt.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=pip.yaml)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=env.yaml)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Download bazelisk v1.15.0] ***************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:55 +0000 (0:00:00.074)       0:00:01.220 ********* 
Step #2 - "build_development_docker_image": ok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Tests] ***********************************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:56 +0000 (0:00:00.846)       0:00:02.066 ********* 
Step #2 - "build_development_docker_image": included: /ansible/roles/bazel/tasks/tests.yaml for localhost
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [bazel : Bazel --version runs succesfully] ********************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:56 +0000 (0:00:00.029)       0:00:02.096 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Add apt keys] *********************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:57 +0000 (0:00:00.582)       0:00:02.679 ********* 
Step #2 - "build_development_docker_image": ok: [localhost] => (item={'url': 'https://apt.llvm.org/llvm-snapshot.gpg.key', 'keyring': '/usr/share/keyrings/llvm.pgp'})
Step #2 - "build_development_docker_image": ok: [localhost] => (item={'url': 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/3bf863cc.pub', 'keyring': '/usr/share/keyrings/cuda.pgp'})
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Add apt repositories into sources list] *******************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:50:58 +0000 (0:00:00.905)       0:00:03.584 ********* 
Step #2 - "build_development_docker_image": ok: [localhost] => (item=deb [signed-by=/usr/share/keyrings/llvm.pgp] http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-17 main)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=deb-src [signed-by=/usr/share/keyrings/llvm.pgp] http://apt.llvm.org/bullseye/ llvm-toolchain-bullseye-17 main)
Step #2 - "build_development_docker_image": ok: [localhost] => (item=deb [signed-by=/usr/share/keyrings/cuda.pgp] https://developer.download.nvidia.com/compute/cuda/repos/debian11/x86_64/ /)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Update apt cache] *****************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:00 +0000 (0:00:02.605)       0:00:06.190 ********* 
Step #2 - "build_development_docker_image": skipping: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install apt packages] *************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:00 +0000 (0:00:00.024)       0:00:06.214 ********* 
Step #2 - "build_development_docker_image": changed: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install pip packages] *************************************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:06 +0000 (0:00:05.576)       0:00:11.791 ********* 
Step #2 - "build_development_docker_image": ok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [install_deps : Install pip packages without deps (--no-deps)] ************
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:08 +0000 (0:00:01.496)       0:00:13.287 ********* 
Step #2 - "build_development_docker_image": �[91m[WARNING]: No valid name or requirements file found.
Step #2 - "build_development_docker_image": �[0mok: [localhost]
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [configure_env : Append environment variables required during runtime to ~/.bashrc] ***
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:08 +0000 (0:00:00.363)       0:00:13.650 ********* 
Step #2 - "build_development_docker_image": changed: [localhost] => (item=CC)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=CXX)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=LD_LIBRARY_PATH)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=TF_CUDA_COMPUTE_CAPABILITIES)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=XLA_CUDA)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": TASK [configure_env : Append environment variables required during runtime to ~/.zshrc] ***
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:09 +0000 (0:00:01.249)       0:00:14.899 ********* 
Step #2 - "build_development_docker_image": changed: [localhost] => (item=CC)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=CXX)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=LD_LIBRARY_PATH)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=TF_CUDA_COMPUTE_CAPABILITIES)
Step #2 - "build_development_docker_image": changed: [localhost] => (item=XLA_CUDA)
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": PLAY RECAP *********************************************************************
Step #2 - "build_development_docker_image": localhost                  : ok=12   changed=4    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
Step #2 - "build_development_docker_image": 
Step #2 - "build_development_docker_image": Wednesday 01 May 2024  19:51:10 +0000 (0:00:01.157)       0:00:16.057 ********* 
Step #2 - "build_development_docker_image": =============================================================================== 
Step #2 - "build_development_docker_image": install_deps : Install apt packages ------------------------------------- 5.58s
Step #2 - "build_development_docker_image": install_deps : Add apt repositories into sources list ------------------- 2.61s
Step #2 - "build_development_docker_image": install_deps : Install pip packages ------------------------------------- 1.50s
Step #2 - "build_development_docker_image": configure_env : Append environment variables required during runtime to ~/.bashrc --- 1.25s
Step #2 - "build_development_docker_image": configure_env : Append environment variables required during runtime to ~/.zshrc --- 1.16s
Step #2 - "build_development_docker_image": Gathering Facts --------------------------------------------------------- 1.12s
Step #2 - "build_development_docker_image": install_deps : Add apt keys --------------------------------------------- 0.91s
Step #2 - "build_development_docker_image": bazel : Download bazelisk v1.15.0 --------------------------------------- 0.85s
Step #2 - "build_development_docker_image": bazel : Bazel --version runs succesfully -------------------------------- 0.58s
Step #2 - "build_development_docker_image": install_deps : Install pip packages without deps (--no-deps) ------------ 0.36s
Step #2 - "build_development_docker_image": Include vars from config files ------------------------------------------ 0.07s
Step #2 - "build_development_docker_image": bazel : Tests ----------------------------------------------------------- 0.03s
Step #2 - "build_development_docker_image": install_deps : Update apt cache ----------------------------------------- 0.02s
Step #2 - "build_development_docker_image": Removing intermediate container 8666ef6ebe69
Step #2 - "build_development_docker_image":  ---> 74b80b3ac315
Step #2 - "build_development_docker_image": Successfully built 74b80b3ac315
Step #2 - "build_development_docker_image": Successfully tagged us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:cuda
Step #2 - "build_development_docker_image": Successfully tagged us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.3
Step #2 - "build_development_docker_image": Successfully tagged us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development:3.10_cuda_12.3_20240501
Step #2 - "build_development_docker_image": Successfully tagged local_image:latest
Finished Step #2 - "build_development_docker_image"
Starting Step #3 - "push_development_docker_image"
Step #3 - "push_development_docker_image": Already have image (with digest): gcr.io/cloud-builders/docker
Step #3 - "push_development_docker_image": The push refers to repository [us-central1-docker.pkg.dev/tpu-pytorch-releases/docker/development]
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Preparing
Step #3 - "push_development_docker_image": b9109473f507: Preparing
Step #3 - "push_development_docker_image": 1060896a6be8: Preparing
Step #3 - "push_development_docker_image": c4f44703eb27: Preparing
Step #3 - "push_development_docker_image": a105bc96d154: Preparing
Step #3 - "push_development_docker_image": 1e15f07af706: Preparing
Step #3 - "push_development_docker_image": 392f916c39cc: Preparing
Step #3 - "push_development_docker_image": 8aec4a886a70: Preparing
Step #3 - "push_development_docker_image": c4c533a462b4: Preparing
Step #3 - "push_development_docker_image": f50eca7ee9ab: Preparing
Step #3 - "push_development_docker_image": 04a325c21dbd: Preparing
Step #3 - "push_development_docker_image": 9dcbabc40b87: Preparing
Step #3 - "push_development_docker_image": 1e15f07af706: Waiting
Step #3 - "push_development_docker_image": 392f916c39cc: Waiting
Step #3 - "push_development_docker_image": 8aec4a886a70: Waiting
Step #3 - "push_development_docker_image": c4c533a462b4: Waiting
Step #3 - "push_development_docker_image": f50eca7ee9ab: Waiting
Step #3 - "push_development_docker_image": 04a325c21dbd: Waiting
Step #3 - "push_development_docker_image": 9dcbabc40b87: Waiting
Step #3 - "push_development_docker_image": a105bc96d154: Layer already exists
Step #3 - "push_development_docker_image": 1e15f07af706: Layer already exists
Step #3 - "push_development_docker_image": 392f916c39cc: Layer already exists
Step #3 - "push_development_docker_image": 1060896a6be8: Pushed
Step #3 - "push_development_docker_image": 8aec4a886a70: Layer already exists
Step #3 - "push_development_docker_image": c4c533a462b4: Layer already exists
Step #3 - "push_development_docker_image": f50eca7ee9ab: Layer already exists
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Pushed
Step #3 - "push_development_docker_image": 04a325c21dbd: Layer already exists
Step #3 - "push_development_docker_image": 9dcbabc40b87: Layer already exists
Step #3 - "push_development_docker_image": c4f44703eb27: Pushed
Step #3 - "push_development_docker_image": b9109473f507: Pushed
Step #3 - "push_development_docker_image": 3.10_cuda_12.3: digest: sha256:f2ebe3f72678f78c9b07b0209365f49ee6ac8a284b2dca917a0806129a525514 size: 2853
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Preparing
Step #3 - "push_development_docker_image": b9109473f507: Preparing
Step #3 - "push_development_docker_image": 1060896a6be8: Preparing
Step #3 - "push_development_docker_image": c4f44703eb27: Preparing
Step #3 - "push_development_docker_image": a105bc96d154: Preparing
Step #3 - "push_development_docker_image": 1e15f07af706: Preparing
Step #3 - "push_development_docker_image": 392f916c39cc: Preparing
Step #3 - "push_development_docker_image": 8aec4a886a70: Preparing
Step #3 - "push_development_docker_image": c4c533a462b4: Preparing
Step #3 - "push_development_docker_image": f50eca7ee9ab: Preparing
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Layer already exists
Step #3 - "push_development_docker_image": 04a325c21dbd: Preparing
Step #3 - "push_development_docker_image": 9dcbabc40b87: Preparing
Step #3 - "push_development_docker_image": b9109473f507: Layer already exists
Step #3 - "push_development_docker_image": f50eca7ee9ab: Waiting
Step #3 - "push_development_docker_image": 1e15f07af706: Waiting
Step #3 - "push_development_docker_image": 1060896a6be8: Layer already exists
Step #3 - "push_development_docker_image": 392f916c39cc: Waiting
Step #3 - "push_development_docker_image": c4f44703eb27: Layer already exists
Step #3 - "push_development_docker_image": 9dcbabc40b87: Waiting
Step #3 - "push_development_docker_image": 8aec4a886a70: Waiting
Step #3 - "push_development_docker_image": c4c533a462b4: Waiting
Step #3 - "push_development_docker_image": 04a325c21dbd: Waiting
Step #3 - "push_development_docker_image": a105bc96d154: Layer already exists
Step #3 - "push_development_docker_image": 1e15f07af706: Layer already exists
Step #3 - "push_development_docker_image": 392f916c39cc: Layer already exists
Step #3 - "push_development_docker_image": 8aec4a886a70: Layer already exists
Step #3 - "push_development_docker_image": f50eca7ee9ab: Layer already exists
Step #3 - "push_development_docker_image": 04a325c21dbd: Layer already exists
Step #3 - "push_development_docker_image": 9dcbabc40b87: Layer already exists
Step #3 - "push_development_docker_image": c4c533a462b4: Layer already exists
Step #3 - "push_development_docker_image": 3.10_cuda_12.3_20240501: digest: sha256:f2ebe3f72678f78c9b07b0209365f49ee6ac8a284b2dca917a0806129a525514 size: 2853
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Preparing
Step #3 - "push_development_docker_image": b9109473f507: Preparing
Step #3 - "push_development_docker_image": 1060896a6be8: Preparing
Step #3 - "push_development_docker_image": c4f44703eb27: Preparing
Step #3 - "push_development_docker_image": a105bc96d154: Preparing
Step #3 - "push_development_docker_image": 1e15f07af706: Preparing
Step #3 - "push_development_docker_image": 392f916c39cc: Preparing
Step #3 - "push_development_docker_image": 8aec4a886a70: Preparing
Step #3 - "push_development_docker_image": c4c533a462b4: Preparing
Step #3 - "push_development_docker_image": f50eca7ee9ab: Preparing
Step #3 - "push_development_docker_image": 04a325c21dbd: Preparing
Step #3 - "push_development_docker_image": 9dcbabc40b87: Preparing
Step #3 - "push_development_docker_image": 4b9d4b7a4e5f: Layer already exists
Step #3 - "push_development_docker_image": 1060896a6be8: Layer already exists
Step #3 - "push_development_docker_image": c4f44703eb27: Layer already exists
Step #3 - "push_development_docker_image": c4c533a462b4: Waiting
Step #3 - "push_development_docker_image": 392f916c39cc: Layer already exists
Step #3 - "push_development_docker_image": f50eca7ee9ab: Waiting
Step #3 - "push_development_docker_image": 8aec4a886a70: Waiting
Step #3 - "push_development_docker_image": 8aec4a886a70: Layer already exists
Step #3 - "push_development_docker_image": a105bc96d154: Layer already exists
Step #3 - "push_development_docker_image": 04a325c21dbd: Waiting
Step #3 - "push_development_docker_image": 9dcbabc40b87: Waiting
Step #3 - "push_development_docker_image": 9dcbabc40b87: Layer already exists
Step #3 - "push_development_docker_image": f50eca7ee9ab: Layer already exists
Step #3 - "push_development_docker_image": c4c533a462b4: Layer already exists
Step #3 - "push_development_docker_image": 04a325c21dbd: Layer already exists
Step #3 - "push_development_docker_image": b9109473f507: Layer already exists
Step #3 - "push_development_docker_image": 1e15f07af706: Layer already exists
Step #3 - "push_development_docker_image": cuda: digest: sha256:f2ebe3f72678f78c9b07b0209365f49ee6ac8a284b2dca917a0806129a525514 size: 2853
Finished Step #3 - "push_development_docker_image"
PUSH
DONE

Build Log: https://console.cloud.google.com/cloud-build/builds;region=us-central1/2827e3f6-1b6b-4b1a-ac95-31ef63be687e?project=1001674285173