Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize dynamic initialization for LRE workers #1485

Open
aaronmondal opened this issue Nov 19, 2024 · 0 comments
Open

Optimize dynamic initialization for LRE workers #1485

aaronmondal opened this issue Nov 19, 2024 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@aaronmondal
Copy link
Member

We currently evaluate the correct toolchain identifier on-the-fly:

args:
- |
git config --global --add safe.directory "*"
NATIVELINK_WORKER_PLATFORM=docker://lre-cc:$(nix eval /mnt/src_root#lre-cc.imageTag --raw) &&
printf '#!/bin/sh\nexport NATIVELINK_WORKER_PLATFORM=%s\nexec "$@"' "$NATIVELINK_WORKER_PLATFORM" > /entrypoint/entrypoint.sh &&
chmod +x /entrypoint/entrypoint.sh

The "on-demand" evaluation was a straightforward approach to achieve strong reproducibility guarantees and to support running workers with different tags side-by-side. However, running the identifier evaluation during initializaion means that it slows down autoscaled workers since all of them have to run through the eval step.

There are many ways to improve the performance here, but we need to be careful not to impact the reproducibility guarantees of the existing setup. We also need to evaluate what the right place to fix this actually is.

@aaronmondal aaronmondal added the enhancement New feature or request label Nov 19, 2024
@aaronmondal aaronmondal self-assigned this Nov 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant